MsDO Weekplan 11

The learning goals for Week 46 are:

Stability Patterns. Replication and Redundancy. Redis Cluster.

Literature:

Slides:

Notes for this weekplan:

The library Resilience4J is a strong library to implement Nygard failure mode handling and I used it in the last instance of the MSDO course. However, the learning curve is steep (it uses a lot of functional programming aspects of Java) and experience from last year showed that it was way too steep. Therefore the slides are only for reference, and in our mandatory, you are asked to hand-code the TimeOut and CircuitBreaker patterns.

Exercise 'timeout-quote-service'

I have uploaded a newer image for the QuoteService, find it at henrikbaerbak/quote:msdo_2_0.

If you run it with the 'slow mode' enabled: 

docker run -d -p 6777:6777 henrikbaerbak/quote:msdo_2_0 java -jar quote.jar 6777 quote.txt slow
you will get the normal quote service behaviour except any upcall takes 20 seconds to complete.

Try to start it and GET Einstein's quote 1: 

> http localhost:6777/msdo/v1/quotes/1

Exercise: Based upon your solution to 'integration-quote-service' from mandatory iteration 3 from the last course, implement the Nygard timeout pattern around the 'getQuote()' method. Parameters must be a timeout on connections of three seconds (fails after 3 seconds if no REST service can be contacted); and a timeout of five seconds on slow responses.

Hint: you can test connection failures by pointing to, say, "localhost:6778" (no service on that port), and of course test slow response failures by pointing to the above service running in 'slow mode'.

Important Design Hint: It is tempting to start recoding your quote service connector implementation. I advice to avoid that, and instead use the Proxy Pattern, to implement safe failure modes in a detached way. That is the call sequence from PlayerServant will pass along this route: 

PlayerServant -> TimeOutProxyQuoteService -> RealQuoteService

That is, the RealQuoteService will throw exceptions, and the proxy catch these, and convert them into an 'graceful degradation quote' that is then returned to PlayerServant, ala 

== Welcome to SkyCave, player Mikkel ==
Entering command loop, type "q" to quit, "h" for help.
> quote 1
*** Quote service not available, sorry. Slow response. Try again later. *** - SkyCave Daemon

Why? Separation of concerns. Avoid bloating the connection code. Support automated testing: You can wrap the TimeOutProxyQuoteService around a Meszaros Saboteur quote service instead that just throws exceptions on every 'getQuote()' call and this way get your time out code under automated testing control!

Exercise 'redis-cluster'

Consult the Redis Cluster Tutorial. It is a long read, but focus on making a cluster work in your VM, which boils down to looking at sections

  1. Creating and using a Redis Cluster. I advice, not to use a config file, but provide parameters on the command line directly, ala issuing this one below 6 times while varying the parameters for portmaps and docker container name: 
    docker run -d --name redis1 -p 7001:7000 --network redis-network redis:6.2.5-alpine redis-server --port 7000 --cluster-enabled yes --cluster-config-file nodes.conf --cluster-node-timeout 5000 --appendonly yes
    which... require that you have 
    docker network create redis-network
    first...
  2. We are running Redis 5+ so skip to the Creating the Cluster section, but issue it using 'docker exec -ti' ala 
    docker exec -ti redis1 redis-cli -p 7000 --cluster create 172.18.0.2:7000 172.18.0.3:7000 172.18.0.4:7000 172.18.0.5:7000 172.18.0.6:7000  172.18.0.7:7000 --cluster-replicas 1
    You will have to verifiy manually that the IP addresses match by entering each container and inspect the locally assigned IP address (issue 'ifconfig' in a 'docker -ti exec (container) sh' shell).
  3. If all is well, you get the [OK] All 16384 slots covered.

Now you can play around with the cluster, from sections Testing the failover or Resharding. I did this without any script writing to the DB, just issued some SET and GET commands.

Next you may try to operate it from Java. Here is my initial program: build.gradle and RedisCluster.java. Copy these files into a proper gradle directory structure, and you should be able to gradle run.

Exercise 'timeout-quote-service-resilience4j'

Code the above exercise, but using the Resilience4J library instead.